Comments/Ratings for a Single Item

Just to give a heads up on what's going to be happening soon, I am
currently working on a login system. I want to get this in place before I
set up the ratings system. At present, you need to enter your userid and
password together each time you need to confirm your identity here. With
the login system, you will be able to log in once and be automatically
recognized without having to enter your password again. I have the CVP
login working already, but I'm also adding OpenID login. This will let you
login with your Facebook, Twitter, Google, Yahoo, or MyOpenID accounts, as
well as through any other OpenID provider by directly typing in your
OpenID. I think I know how to make it work now, but I still need the time
to work on it. When you use either login, it will store cookies used to
recognize you. One cookie will tell who you are, and another will be an
encrypted string used to verify the accuracy of the first cookie. You will
be able to connect an OpenID with your CVP account, which will let you
directly login to your CVP account through an OpenID provider. This will
come in handy if you forget your CVP password, but it will normally be
faster to login to your CVP account directly. Another use I plan to make of
OpenID is to make registration faster. If you login with an OpenID, you
will be able to register without email authentication. As I envision things
working, you will be able to use an OpenID or a CVP userid to post
comments, but you will need a CVP account to rate games, vote in polls, or
play games on Game Courier.

haha, someone rating their own game is exactly like parents rating their
own kids, and no one knows a child best like their parents, but everyone
else just smiles.

Ben Reiniger wrote:

I'm a little worried about rating games when they are themed or experimental: for instance, my only game is 4D, so probably not a good candidate for frequent play, but (IMO of course) it is a very good setup as far as 4D games go. Should it get rated in the former sense or the latter? Maybe the frequency of play question will serve this purpose too, but maybe there should be a third rating? (So 'good game', 'interesting', and 'often played'?)

Yes, the frequency of play question should serve this purpose. It will help separate dislikes from people who are just prejudiced against 4D games without playing your particular game from dislikes by people who have actually played it. In general, ratings from people who have played a game will matter more than ratings from people who have not played a game. That should help protect niche games from being underrated.

Joe Joyce wrote

First, maybe separate the ratings into 3 groups: author rating, verified raters, unverified raters, then show them in order of best rating to worst rating.

There won't actually be unverified raters, because each rating will be keyed to the userid. This prevents ballot stuffing, and it allows raters to change their ratings. I plan to include inventor ratings in the header when they are available, along with figures on the other ratings.

Second, use both types of words for rating. I can see rating a game excellent, but not a favorite, or a game average, but a favorite.

I don't want to overcomplicate things. One of my main goals with a new rating system is to make it easy to use. One possibility is to integrate the new and old rating systems, such that if someone rates a game as Poor or BelowAverage in a comment, it gets recorded as a Dislike, and if someone rates a game as Good or Excellent, it gets recorded as a Like. But a user would still be free to change his ratings in the new system, even favoriting a game he has rated as average in a comment.

Third, group games by both clustering and category. Dimensionality would be one obvious category, as would size, but I would suggest adding a few groupings and categories that aren't differentiated here.

Since this information is already in the database, and ratings will be keyed to ItemIDs, it should be a simple matter to compare the ratings for groups of related games.

The purpose of all this is to direct people to games they might like quickly. We have around a thousand presets now, and a few times that in total variants. If this is to be useful, we should offer patrons a useable search mechanism. Is that possible with the resources we have?

Here are a few things I plan on adding: 1) People who like this game also like these games. 2) Given your likes and favorites, here are other games you may like.

I've been following this conversation with a little interest for a while.
Might I offer a few [undoubtedly complicating] suggestions? 

First, maybe separate the ratings into 3 groups: author rating, verified
raters, unverified raters, then show them in order of best rating to worst
rating. 

Second, use both types of words for rating. I can see rating a game
excellent, but not a favorite, or a game average, but a favorite. Let me
use Ben's game, TessChess, as an example. It is an excellent 4-D chess
variant. It may not be the best fully 4-D FIDE variant possible, but it is
certainly one of them. It is, unfortunately, essentially unplayable.
Computer assistance would be very beneficial to this game by showing all
the moves a piece could make, but the game would still be rather difficult.
Ben and I have discussed our 4-D games, and essentially agree on matters of
dimensionality and playability. HyperChess is not fully 4-D, but that makes
it humanly playable without aid. It even has a few fans. 

Third, group games by both clustering and category. Dimensionality would be
one obvious category, as would size, but I would suggest adding a few
groupings and categories that aren't differentiated here. There are a lot
of games that would fall under the [Los Alamos -] FIDE - Capablanca - Grand
Chess cluster, and could be called the 'pure FIDE' cluster, as they use
only the elements of FIDE. The Chinese variants, XiangQi, Janggi... are
another cluster, as are the shogis. But the shogis are also 'Drop' games,
games that bring reinforcements onto the board at some point[s] in the
game. Grin, and, at this point, shatranj variants would be another game
cluster, associated with a 'short range' category, and thus cousins of
the shogis, which also feature short ranged pieces.

The purpose of all this is to direct people to games they might like
quickly. We have around a thousand presets now, and a few times that in
total variants. If this is to be useful, we should offer patrons a useable
search mechanism. Is that possible with the resources we have?

Ben Reiniger wrote:

I think the inventor shouldn't rate the game; if they want to mention what they think of the game, it should go into the page somewhere (in the introduction would be good).

Judging by the data I collected on the frequency of ratings in the comment system, there is a lot of rating inflation going on, and it can't all be due to inventors rating their own games more favorably than more objective critics. In fact, I can screen for that. Here are the results of non-anonymous ratings by people other than the inventor:

Array
(
    [Excellent] => 782
    [Good] => 698
    [Poor] => 137
    [Average] => 27
    [BelowAverage] => 18
)

Excellent is still the most common rating, followed by Good and Poor. We can also look specifically at how inventors rate their own games. Here are the results of non-anonymous raters rating their own games:

Array
(
    [Excellent] => 18
    [Good] => 13
    [Poor] => 3
    [Average] => 2
    [BelowAverage] => 1
)

This is a smaller sample size, but the pattern is not significantly different. Pride might motivate inventors to rate their own games higher, but that doesn't account for others also giving their games an Excellent rating more often than other ratings. Perhaps the other raters feel they would be hurting the inventor's feelings if they rated a game anything less than Excellent. Perhaps the word 'excellent' has lost its comparative connotations. We may use the word too liberally, even for things that aren't really that much better than other things.

One of my main concerns in designing a new rating system is to avoid the rating inflation common in the current rating and comment system. One thing I think will help is to replace qualitative rating terms (Poor, BelowAverage, Average, Good, Excellent) with personal preference terms (Dislike, Like, Favorite). I think Favorite works better than Excellent, because it is more clearly a comparative term than Excellent is. Judging from the data I just presented, I don't think that banning inventors from rating their own games would do anything to reduce rating inflation.

Furthermore, given that an inventor is usually the most experienced player of a Chess variant, or at least the one person who knows the game better than anyone else does, I think the opinion an inventor has of his own games is valuable information, and it should be included in the system.

Fergus Duniho writes:

What kind of performance issues are there? With a composite primary key, will it search individual keys until it finds a match instead of directly knowing where to go?

It is quicker to traverse a numeric key than an alpha or alpha-numeric. But it is not just a question of machine performance - development time including code production (manual and automated) and testing can be less efficient with composite primary keys. There is also the question of maintenance and the persistence of natural composite keys when compared to surrogate primary keys.

But this isn't the place to conduct a seminar in database design and I have already declared myself guilty of misapplying enterprise practice - it isn't easy, sometimes, to put aside work practices gleaned over 3 decades.

I had a script count the number of times each rating has been used in the
comments. Here are the results:

Array
(
    [Good] => 2052
    [None] => 21618
    [Excellent] => 3167
    [Poor] => 511
    [] => 90
    [Average] => 156
    [BelowAverage] => 76
)

I then thought to do the same thing on only Game pages. Here are the
results of that:

Array
(
    [Good] => 1507
    [None] => 9939
    [Excellent] => 2068
    [Poor] => 342
    [BelowAverage] => 43
    [Average] => 69
    [] => 19
)

Whichever results I use, the same pattern emerges. Discounting None and
null, Excellent is used most often, followed by Good and Poor. BelowAverage
and Average are used much less than these three. This supports the idea
that most raters are really employing a three-point scale rather than a
five-point scale. This lends support to using the 'Dislike, Like,
Favorite' scale over the five-point scale currently in use with the
comments.

Judging from this data, I also think that Excellent has been overused. More
games have been rated Excellent than the sum of all the other ratings. A
rating of Excellent should be reserved for the very best games, and if it
were, it would be a less common rating. The way I want to use Favorite may
curb this tendency. One thing I want to do with favorites is list each
user's favorites on his information page. When selecting a game as
favorite, the user would understand that he is also adding it to a viewable
list of favorites, and he may choose his rating with a view to this.

For the rating scale, here are some options I have in mind:

Dislike, Like, Favorite

Poor, Inferior, Good, Superior, Exceptional

Dislike, Interesting, Like, Favorite

Here are three different options:

1) Keep the rating and comment system separate from the dedicated game
rating system I have in mind.

2) Integrate them.

3) Turn the rating and comment system into just a comment system. Keep all
the old ratings in the comment system but don't allow new ratings. Export
the ratings of game pages into the dedicated game rating system.

I generally like the idea.  I think the inventor shouldn't rate the game;
if they want to mention what they think of the game, it should go into the
page somewhere (in the introduction would be good).  I'm a little worried
about rating games when they are themed or experimental: for instance, my
only game is 4D, so probably not a good candidate for frequent play, but
(IMO of course) it is a very good setup as far as 4D games go.  Should it
get rated in the former sense or the latter?  Maybe the frequency of play
question will serve this purpose too, but maybe there should be a third
rating?  (So 'good game', 'interesting', and 'often played'?)

David Howe wrote:

So my feeling is that we should have a single, consistent commenting and rating system. Having two parallel systems is confusing and inelegant. That said, I am at a loss for how to proceed. Somehow, we should keep what we have, but also add desired functionality, but as a single, comprehensive system.

I suppose we could export the existing comment and ratings into the proposed new system, and have a unified and elegant commenting and rating system. That will take more effort, but may be worth it.

That could be tricky. For one thing, comment ratings are associated with individual comments instead of with individual users. If I post two comments to a page with different ratings, I now have multiple ratings for the same game. If we exported the comment ratings, we would have to identify the latest ratings made by each user. Also, what I have in mind is specifically for games, not for pages. The rating and comment system in place is available for all pages, and even on game pages, people sometimes use it to rate the page instead of the game itself. Although I have previously proposed using a five-point rating scale, I have more recently been thinking of going with Like, Dislike, and Favorite. There is also the matter of measuring popularity, which the rating and comment system does not handle at all.

Assuming I used a five-point scale, the ratings from the rating and comment system might be incorporated into the Rating field of the Likes table. When someone posts a comment, it could show his current rating for the game, and if he changed it with his comment, it would update the Rating value in the Likes table. A comment could also report the player's experience with the game at the time of the comment. So the comment table would keep track of historical data, and the Likes table would keep track of current data. The ratings in the comments could be replaced by integers, and a separate table could be used to translate them into words. Besides entering ratings with a comment, I would provide a rating form in the header, which can be used without entering a comment, and it would just update the Likes table.

Those are my thoughts on how they could be merged together. I'm still not sure it would be the best way to go. We may want to keep them separate, using one for page ratings and one for game ratings.

I'll let you know if I need help with SQL.

Fergus, if you need any help with the SQL end of things, please let me know.

There are two existing rating and commenting systems. I will refer to the older (and heavily used) system as the rating and commenting system. The newer system, which is not used to any significant extent, I will call the newer rating system.

I agree that the newer rating system is not used (much to my chagrin, given the work I put into it) and should be wiped from the site. The older rating and commenting system seems to work acceptably, although it has its issues and limitations. I agree that we shouldn't toss years of valuable comments/ratings.

So my feeling is that we should have a single, consistent commenting and rating system. Having two parallel systems is confusing and inelegant. That said, I am at a loss for how to proceed. Somehow, we should keep what we have, but also add desired functionality, but as a single, comprehensive system.

I suppose we could export the existing comment and ratings into the proposed new system, and have a unified and elegant commenting and rating system. That will take more effort, but may be worth it.

Just my 2 cents. Please let me know if I can be of any help on the PHP or SQL end of things. I really hope we can achieve a more effective and elegant system than the existing one.

Graeme Neatham wrote:

Primary Key - the Candidate Key that is the main index for a table and which will be used as a Foreign Key in other tables. For performance purposes a Primary Key will usually not be Composite.

What kind of performance issues are there? With a composite primary key, will it search individual keys until it finds a match instead of directly knowing where to go?

Self-rating - personally I would be wary of such a practice, not because I think inventors are always going to give maximum marks to their own games, but because they are too close to their own games to provide a disinterested assessment.

I'm not worried about that. I think that game inventors, especially the more productive ones, distinguish between games they consider experiments and games they consider masterpieces. And especially when it comes to inventors of several games, I want to know which ones they like best. Regardless of how objective inventors can be in evaluating their own games, I want to know how they evaluate their own games.

Overwriting of data due to revision of values leading to loss of historical data strikes me as being what would be called 'bad practice'. But I guess I'm guilty of trying to apply corporate standards in a non-corporate situation.

I'm not worried about overwriting old data, because it will have no use anyway. Also, if I included old data, I would either have to expand the table dynamically or use a more complicated primary key. The latter approach could be a real problem, because the php script would not know what primary key to use to access the ratings.

Relational Database terminology:

• Candidate Key - a column or set of columns providing data that is unique for each row. Where a set of columns is involved the term Composite Candidate Key is used.
• Primary Key - the Candidate Key that is the main index for a table and which will be used as a Foreign Key in other tables. For performance purposes a Primary Key will usually not be Composite.
• Foreign Key - the Primary Key of another table.
• Key - when used without qualification will normally be understood to be shorthand for Primary Key

Self-rating - personally I would be wary of such a practice, not because I think inventors are always going to give maximum marks to their own games, but because they are too close to their own games to provide a disinterested assessment.

Overwriting of data due to revision of values leading to loss of historical data strikes me as being what would be called 'bad practice'. But I guess I'm guilty of trying to apply corporate standards in a non-corporate situation.

Having read some more about MySQL, I think the table might look like this:

create table Likes (
    ItemID blob,
    UserID blob,
    Freq tinyint,
    Rating tinyint,
    primary key (ItemID, UserID),
    key (ItemID),
    key (UserID)
);

I removed UniqID, because I learned that a key can be made from multiple
fields.

Graeme Neatham wrote:

It might be useful to include an ID for the item's creator. I know this can be found by joining through the ItemID but keeping it in this table as well would enable faster queries concerning games inventors. It would also make it easier to stop creators rating their own games (UserID != CreatorID)

Actually, I want to encourage inventors to rate their own games. A game's inventor is usually the most experienced player of the game. As an inventor myself, I'm not expecting inventors to give top ratings to all of their games. I would give top ratings to some of my games and not to others. I don't think there is any need to add inventor ids to the table, because these can be retrieved easily enough from the Item table.

Will a user be able to revise their ratings? If so the table will need a timestamp field and possibly either a binary current/superseded field or a status field. Or previous ratings might have their own table

Yes, users will be able to revise their ratings. I don't see the need for a timestamp or other field. Old rating values will simply be replaced with new rating values.

Just a few thoughts about the Likes table:

• It might be useful to include an ID for the item's creator. I know this can be found by joining through the ItemID but keeping it in this table as well would enable faster queries concerning games inventors. It would also make it easier to stop creators rating their own games (UserID != CreatorID)
• Will a user be able to revise their ratings? If so the table will need a timestamp field and possibly either a binary current/superseded field or a status field. Or previous ratings might have their own table - LikesHistory

We have that already. It is called Recognized Variants. But it takes effort
to add to it, and it is not as responsive to user preferences.

For the technically inclined, here is the table I'm thinking of adding:

create table Likes (
    UniqID blob,
    ItemID blob,
    UserID blob,
    Freq tinyint,
    Rating tinyint,
    primary key(UniqID),
    key (ItemID, UserID)
);

The UniqID will be a concatenation of ItemID with UserID, and its purpose
is to uniquely identify a row for quick reading and writing. The other two
keys, ItemID and UserID, may have the same value for multiple rows and are
included for finding all rows with the same ItemID or the same UserID. The
remaining two values include integers representing answers to the two
questions.

you know, another idea, besides what you doing, if you are worried about
people coming to this site and not finding quality games, or finding it
hard to find them, making a link on main page to 'recommended games', or
something along those lines (some recommended top quality games) etc etc. 

You could start by just picking some obviously top games. Later you could
change it to link to the top rated games that have been rated by your
system.

cool, well done, hehe, good work, just you know, being protective and
silly, great work fergus, your a legend.
sorry if i cause trouble, hehe, dont mean to.

Christine Bagley-Jones wrote:

But i see you are bent on changing things, without bothering also to talk to the other members apparently, haha.

Talking to the other members about what I'm planning is exactly what I'm doing here. But so far, you're the only one who has had anything to say about it. I assume from their silence that other members are okay with what I'm planning.

The only thing i am trying to do is keep 10 years of members ratings.

I plan to leave the comments sections and page ratings alone. What I intend to replace is the game ratings system that has been used for only three games the whole time it has been here.

Every game has a game page, every member can rate these pages. Sounds fair to me, i dont see how that is unfair to any game. I have faith in the members rating games, even non-members, for that matter.

It's fair in that it is equally useless for all games. While it allows people to rate and review games, it does not make use of this information in any way. What I have planned is a dedicated game rating system that can report back various types of information, such as the popularity or average rating for a game, an individual member's favorite games, or whether the inventor has played the game and how he rates it himself.

Can i ask, what does it mean to put ratings in the 'item table'?

It is MySQL jargon, and only David Howe needs to understand precisely what I mean. Anyhow, I'm now thinking it would be best to create a new table in the database, because it would be shorter than the Item or Person tables and so faster to access.

Is this a page like this .. http://www.chessvariants.com/index/mainquery.php?type=Any&startswithletter=Na&orderby=LinkText&displayauthor=1&displayinventor=1

No, it's nothing you would see on the website.

Fergus, you said 'What you're wanting to do is minimize false negatives,
quality games that unfairly go unrecognized.' Sorry, this is not true. The
only thing i am trying to do is keep 10 years of members ratings. Every
game has a game page, every member can rate these pages. Sounds fair to me,
i dont see how that is unfair to any game. I have faith in the members
rating games, even non-members, for that matter.
But i see you are bent on changing things. Can i ask, what does it mean to put ratings in the 'item table'? Is this a page like this ..
http://www.chessvariants.com/index/mainquery.php?type=Any&startswithletter=Na&orderby=LinkText&displayauthor=1&displayinventor=1

While trying to figure out how to make a new ratings system work, I have
been looking into the database to see what is there. One of the tables is
called Ratings, and it is for the rating system I have described as hidden
and useless. Only three games have been rated. These are an example game
David Howe made to test the ratings system, European Chess, and Shatranji.
Given the scant use of this ratings system, I think it would be alright to
discard it. What I want to replace it with will be more prominent and
easier to use, making it more likely that people will use it. What I'm
trying to decide on right now is how to represent the data in the database.
It could go in the Item table, the Person table, or both. Putting it in the
Item table makes it easier to calculate the average ratings for a game,
while putting it in the Person table makes it easier to list someone's
favorites. I'll probably go with putting it in the Item table. Since
userids are uniformly lowercase and all the keys in the Item table are
mixed case, it should be safe to add userids as keys in the Item table. The
userid would be a key to a two-character string representing how someone
has answered the two questions. To calculate an average rating, I could
then scan an Item row for lowercase keys, add up the values and divide.